Assessing patient risk, benefit, and outcomes in drug development: A decade of ramucirumab clinical trials

Abstract Objective This study aims to evaluate published clinical trials of ramucirumab to assess the risk/benefit profile and burden over time for patients. Background The burden of oncologic drug development on patients paired with increasing clinical trial failure rates emphasizes the need for reform of drug development. Identifying and addressing patterns of excess burden can guide policy, ensure evidence‐based protections for trial participants, and improve medical decision‐making. Methods On May 25, 2023 a literature search was performed on Pubmed/MEDLINE, Embase, Cochrane CENTRAL, and ClinicalTrials.gov for clinical trials using ramucirumab as monotherapy or in combination with other interventions for cancer treatment. Authors screened titles and abstracts for potential inclusion in a masked, duplicate fashion. Following data screening, data was extracted in a masked, duplicate fashion. Trials were classified as positive when meeting their primary endpoint and safety, negative or indeterminate. Results Ramucirumab was initially approved for gastric cancer but has since been tested in 20 cancers outside of its FDA approved indications. In our analysis of ramucirumab trials, there were a total of 10,936 participants and 10,303 adverse events reported. Gains in overall survival and progression‐free survival for patients were 1.5 and 1.2 months, respectively. FDA‐approved indications have reported more positive outcomes in comparison to off‐label indications. Conclusion We found that FDA‐approved indications for ramucirumab had better efficacy outcomes than non‐approved indications. However, a concerning number of adverse events were observed across all trials assessed. Participants in ramucirumab randomized controlled trials saw meager gains in overall survival when evaluated against a comparison group. Clinicians should carefully weigh the risks associated with ramucirumab therapy given its toxicity burden and poor survival gains.

Gastric cancer remains a significant global health burden with high mortality and morbidity rates, particularly in advanced stages of the disease. 1 Despite advancements in gastric cancer treatment there remains a need for new therapeutic approaches for disease management.Ramucirumab, a human monoclonal antibody targeting vascular endothelial growth factor receptor 2 (VEGFR2) emerged as a potential treatment option.4][5] Moreover, patients must assess the risk, benefits, and barriers of trial participation. 68][9] With less than 1 in 20 adult cancer patients enrolling in clinical trials, it remains relevant for all stakeholders to examine the total patient burden of trial participants. 6he high price of oncologic drug development coupled with high clinical trial failure rates have emphasized the need for initiatives to improve the pace of drug development. 10Yet, despite the push for accelerated drug development-it remains unknown what areas pose the most burden to patients.Recent studies have demonstrated the need for researching total patient burden throughout a drug's cumulative life cycle.Carlisle et al. report that following the initial FDA approval of imatinib, subsequent trials evaluating new drug combinations and indications were significantly less likely to attain FDA approval. 11dditionally, Carlisle et al. found worsening risk/benefit ratios as sunitinib clinical trials matured-with numerous negative, replicative trials. 12The risk/benefit variation throughout drug development raises ethical concerns for patient safety and research waste.
Identifying patterns of excess burden in drug development programs can improve research decision-making, guide policy, and ensure evidence-based human protections for trial participants. 12Given the ethical obligation to minimize risk and the need for novel cancer therapies-it is imperative to evaluate drug programs in their entirety and maximize the efficiency of clinical translation.Previous studies have explored the impact of ramucirumab on patient survival in Phase II/III trials but presently, no study has analyzed the risk/benefit profile for ramucirumab used in the treatment of advanced lung, gastric, liver, and colorectal cancers in Phase I-III trials. 13,14Thus, our primary objective was to quantify the total risk/benefit profile for ramucirumab clinical trials and characterize areas of excess burden.

| Study design/openscience
We did a cross-sectional study exploring clinical trials of ramucirumab (trade names Cyramza® or Monoclonal Antibody HGS-ETR2®, Eli Lilly and Company) for their risk/benefit profiles throughout their development and applications to indications beyond initial approval.To improve rigor and reproducibility, as well as promote open science, we uploaded a protocol a priori to the investigation.Following study completion, we uploaded raw data, statistical analysis scripts, and extraction forms to Open Science Framework (OSF)-a free-to-upload data repository. 15Our data are accessible on OSF throughout the repository's lifecycle, or alternatively, it can be obtained by making a request.A red strikethrough and clarifying red text on OSF represented protocol amendments and erratum.

| Research questions, definitions, and hypothesis
Given that clinical trials are costly and potentially harmful for patients, we aimed to evaluate the risks and benefits compared to the efficacy of ramucirumab in clinical trials.Our objective was to evaluate if the combined risk profiles represented an overall excessive risk to patients.A clinical trial profile was defined as the overall risk and benefit encountered by participants during a single trial as measured by selected tools mentioned in the Data Extraction section.A drug's portfolio was defined as the complete compilation of trial profiles for a given intervention.We hypothesized that the inclusion of off-label indications in the expansion of clinical trials for ramucirumab would lead to a higher occurrence of negative trials with elevated patient risk, resulting in an overall negative drug portfolio.

| Training
Author VP, an experienced clinical oncologist and expert in evidence-based medicine, provided training to all authors on clinical trial design, reporting, and outcomes.Authors were educated on both the Common Terminology Criteria for Adverse Events (CTCAE) 16 and Response Evaluation Criteria in Solid Tumors (RECIST). 17Screening authors were trained to use Rayyan (https:// www.rayyan.ai/ ) 17 : an online tool to screen large samples of literature.A pilottested Google extraction form was used for data extraction.Authors extracted and reviewed five studies from the sample that met the inclusion criteria.This preliminary step allowed the authors to train in the extraction process and familiarize themselves with the methodology before proceeding in data extraction.

| Literature search
We performed a literature search on May 25, 2023 of PubMed/MEDLINE, Embase, Cochrane CENTRAL, and Clini calTr ials.gov for clinical trials using ramucirumab as monotherapy or in combination with other interventions for cancer treatment.Our search strings were standardized across these databases using the PolyGlot Search Translator (https:// sr-accel erator.com/#/ polyglot) developed by the Institute for Evidence Based Healthcare and Bond University. 18Our search strings, including date of search and initial returns, were uploaded to OSF.They can be accessed as supplementary data in the final manuscript submission.

| Selection process
We uploaded search returns into Rayyan for literature screening.Two authors (AK and HK) screened titles and abstracts for potential inclusion in a masked duplicate fashion.After screening was complete, author CL was available to resolve any discrepancies.We recorded reasons for exclusion during the screening process.This allowed us to create a flowchart for study exclusion.

| Inclusion and exclusion criteria
The inclusion criteria we used in the screening process evaluated clinical trials that assessed: (1) adult and human subjects, (2) the efficacy of ramucirumab as a monotherapy or in combination with other agents to treat oncological malignancies, (3) the benefit of ramucirumab using radiographically derived response criteria for solid tumors (e.g., RECIST, mRECIST) and laboratory-derived response criteria for nonsolid tumors, and (4) journals published in English.Non-oncological studies, studies on nonsolid tumors, studies on biosimilars, pharmacology studies on healthy participants, and studies of exclusively pediatric populations were excluded.Studies that represented article types that are not clinical trials and non-manuscript publication types including: secondary reports, interim results, clinical trial updates and follow-ups, preclinical studies, literature reviews, systematic reviews, meta-analyses, human tissue studies, laboratory studies, case reports, letters to the editor, editorials, opinion pieces, conference abstracts, corrections, or redactions were excluded.

| Data extraction
Following literature screening, data extraction was performed on the clinical trial sample.Data extraction was done in a masked, duplicate fashion by two authors (AK and HK) with a third author (CL) available to resolve discrepancies.Authors extracted the following variables: published trial title, PubMed ID, clinical trial registry number, country of first author's affiliation, date of publication, metastatic or nonmetastatic stage, number of male participants, number of female participants, single-center or multicenter, indication(s) of the trial, number of participants, mean or median age of participants, randomization ratio, study sponsor including conflicts of interest statements and funding, whether the trial was controlled, if the trial assessed monotherapy or combination therapies, phase of the trial, and blinding of trial participants.
For risk and benefit outcomes the following variables were extracted for treatment arms: the name of the arm, median progression-free survival (PFS) in months, median overall survival (OS) in months, adverse events grade, partial response, complete response, objective response rate (ORR) as defined in the RECIST criteria, number of adverse events (Grades 3-5) as defined in the CTCAE criteria, and if the trial was positive, negative, or indeterminate.Using a prespecified indication, outcome measurements and adverse events encompassing all trial participants were extracted.A trial was classified as positive if it successfully achieved its prespecified endpoints while using a regimen that was well-tolerated.On the other hand, a trial was categorized as negative if it failed to meet its prespecified endpoints or if it utilized a regimen that was not welltolerated.An indeterminate classification was assigned to trials that did not prespecify endpoints and utilized a regimen that was well-tolerated.The tolerability of a regimen was determined by trial authors. 12everal design decisions were implemented concerning the specific characteristics of the trial.If multiple phases were reported in a trial, the higher phase was extracted.Exclusively partial responses were assumed when complete and partial were not specified.Further, when both confirmed and unconfirmed responses were reported, only the confirmed data was extracted.In dose-escalation and dose expansion trials all arms were pooled for analysis.Mutations have been reported by their mutation classification.In crossover studies, all pertinent data were extracted prior to crossover to ensure accuracy.If a trial enrolled subjects in more than one indication then it was reported as "multiple indications."A table detailing such trials was included in the supplementary data.Objective response rate was calculated for evaluable patients; however, if the trialist did not specify then all patients from the trial were included in the calculation.

| Statistical analysis
A descriptive statistics in R (version 4.2.1) and RStudio was conducted.

| Ethical oversight
Our protocol was reviewed by The Oklahoma State University Center for Health Sciences and it was determined that this research qualifies as nonhuman subjects research as defined in regulation 45 CFR 46.102(d) and (f).

| General characteristics
Our initial search identified 1911 studies for inclusion from bibliographic databases.The search on Clini calTr ials.gov found 208 additional trials for inclusion.After deduplication, abstracts and titles were screened, with 153 publications available for full-text screening.Following full-text screening 90 trials were further excluded, yielding a final sample of 63 publications (Figure 1).

| Endpoints
The most common endpoints in our sample were safety and PFS.Progression-free survival was the primary endpoint in

| Delta PFS and OS
Twenty-three randomized controlled trials were analyzed for ΔPFS and ΔOS.The median change in the randomized controlled trials for OS was 1.5 months and PFS was 1.2 months between the ramucirumab treatment arm and the comparison arm.The highest ΔPFS (7.0 months) was in a trial for non-small cell lung cancer that led to FDA approval.FDA-approved indications reported statistically significant ΔPFS values when evaluated against a comparison group.Pancreatic cancer, an off-label indication, had the lowest ΔPFS (−1.1 months) and ΔOS (0.6 months) (Table 5).

| Risk assessment
In our analysis of ramucirumab trials, there were a total of 10,936 participants and 10,303 adverse events reported.A spike in adverse events observed in 2014-2015 occurred after publication of Phase III trials RAINBOW, REVEL, ROSE/TRIO-12, RAISE, and REACH, which enrolled a combined 5342 patients and had 4927 cumulative Grade 3-5 adverse events.An additional peak was seen in 2019 following Phase III trials RAINFALL and RELAY, which enrolled 1094 patients and had 1285 cumulative Grade 3-5 adverse events.Adverse events increased with participant enrollment until 2020.However, since then, this ratio has demonstrated a decline, as seen in Figure 2. As shown in Figure 3, the cumulative adverse event rate (AER) was over 100%, illustrating more adverse events than patients, following the RAINBOW trial in 2014.Following ramucirumab testing for NSCLC, HCC, and gastric cancer from 2010 to 2013, indication use began to be expanded.A metastatic melanoma and ovarian cancer trial conducted in 2014, which had response rates of 10.8% and 5%, brought down cumulative ORR to the lowest values in ramucirumab's trial history.The same year, the highest cumulative ORR was observed, likely due to the ROSE/TRIO-12 and RAINBOW trials which reported ORR values of 42.4% and 22%, respectively.The AER, in relation to the cumulative ORR, remains elevated compared to pre-initial FDA-approval levels, suggesting an increase in overall risk.

| Accumulating evidence and research organization (AERO) diagram
Figure 4 provides a visual representation of the ramucirumab clinical trial portfolio, capturing the progression of its development through different indications and phases.Ramucirumab has had a total of six FDA approvals for four indications with the first approval occurring in 2014 and the most recent in 2020.Advanced gastric cancer and non-small cell lung cancer were approved in 2014, metastatic colorectal cancer in 2015, and advanced hepatocellular carcinoma in 2019.
After 2014 there was an expansion into 21off-label trials, and it is notable that 47% (10 out of 21) of these trials yielded negative results, with none of them receiving FDA approval.Overall, the percentage of positive trials for FDA-approved indications was 21.4% greater than the percentage of non-FDA approved indications, suggesting superior efficacy outcomes for on-label trials.There were 51 Phase I and II studies in our sample, and 12 Phase III studies, indicating limited progression past Phase II.Additionally, Figures 5 and 6 provide a visual representation of cumulative Grade 3-5 events and cumulative ORR in Phases 1-3 of both combination and monotherapy trials.Ultimately, our results indicate that, for clinical trials of both FDA-approved and non-FDA approved indications of ramucirumab, there is an increase in risk for patients over time.However, FDA-approved indication trials of ramucirumab are associated with prolonged survival when compared to off-label, or novel, indication trials.

| DISCUSSION
Ramucirumab is a next in class monoclonal antibody targeting the VEGFR-2. 19It initially received FDA approval in 2014 for the treatment of metastatic gastric cancer in the second line setting and most recently received FDA approval in 2020 for the first-line treatment of metastatic EGFR-mutated non-small cell lung cancer.Since then, the drug has gained regulatory authorization in other metastatic tumor types, specifically non-small cell lung cancer, hepatocellular carcinoma, and colorectal cancer.Out of the 24 total indications in our study, 4 were FDA approved.We sought to put the regulatory authorization in the broader context of the clinical trials portfolio of this drug.
In our study, we performed an in-depth evaluation of clinical trials focusing on the use of ramucirumab to treat solid tumors published within the last 13 years.Our results showed a varied response when investigating ramucirumab beyond its original FDA-approved indications for gastric and gastroesophageal carcinoma.We also observed a significant increase in cumulative Grade 3-5 adverse events between the years 2014 and 2017, accompanied by a proportional decrease in ORR.This finding coincides with several studies exploring off-label use of ramucirumab.
Patients treated with ramucirumab experienced a modest gain of 1.5 months in median OS.This result may indicate the limited effectiveness of ramucirumab in enhancing survival rates.Nonetheless, it is notable that the change in OS and PFS still demonstrate higher values for FDAapproved indications of ramucirumab than off-label indications.Effing and Gyawali, observed gains in median overall survival ranging from 1.2 to 2.2 months and determined these were marginal improvements in relation to the toxicity experienced by patients. 20These findings emphasize the importance of carefully evaluating the efficacy of ramucirumab by clinicians when making treatment decisions.
Additionally, FDA-approved indications for ramucirumab exhibited higher ORR, PFS, and OS when compared to non-approved indications.Grabowski et al. found benefits when ramucirumab was used as treatment for HCC, NSCLC, colorectal cancer, and gastric cancer. 21owever, following ramucirumab's FDA approval in 2014, trialists began investigating its efficacy in various forms of cancer.Our results revealed a wide range of outcomes, with trials yielding both positive and negative effects.For example, multiple studies were conducted examining the efficacy in treatment for both biliary tract cancer and metastatic breast cancer.Unfortunately, these studies have produced predominantly negative responses with trialists acknowledging efficacy outcomes were insufficient for treatment. 22,23Studies conducted by Carlisle et al. on imatinib and sunitinib concluded that indications beyond the scope of FDA approval were only tested after the strongest indications were thoroughly examined and these trials were associated with more negative results. 11,12Our results align with these findings, as testing for indications without strong preclinical evidence, such as breast cancer and ovarian cancer, were associated with more negative trial results and occurred once FDA approval for ramucirumab's initial indications were thoroughly examined.As a VEGFR-2 inhibitor, ramucirumab inhibits angiogenesis, an important mechanism for many tumors. 24I G U R E 2 Cumulative adverse events and participants of ramucirumab trials over time.Neutralizing tumor progression, and extending life, are important efficacy markers for cancer therapeutics.Although our results indicate over a third, or 23 of 36, of trials were negative, nevertheless for certain indications, notably gastric cancer and NSCLC, ramucirumab was found to be associated with prolonged OS and PFS.Specifically, ramucirumab was associated with a higher PFS, OS, and ORR in combination therapy than monotherapy.Given that the majority of ramucirumab's FDA-approvals are for its treatment as a combination therapy, this finding is to be anticipated.Ramucirumab has been found to enhance inhibitory effects in combination, allowing for more efficacious tumor containment. 25,268][29] While clear benefits exist in regards to this treatment's efficacy, understanding the toxicity profile is important to take into consideration as well.
Further, our study revealed a notable risk period in the development of ramucirumab, with the years 2014-2017 exhibiting the highest cumulative Grade 3-5 adverse events and proportionally lowest cumulative ORR.Specifically, a significant contributor to this occurrence was a 2014 study that examined the efficacy of ramucirumab as a combination therapy for the treatment of gastric or gastroesophageal junction cancer.This study reported 72 total Grade 5 events in 647 patients.It is important to recognize that a case-specific investigation is warranted before attributing the cause of these Grade 5 events directly to the toxicity of ramucirumab.Despite a low ORR of 16%, the trial was classified as positive as it met the prespecified primary endpoint of median OS even with a high AER.The authors concluded the toxicity was acceptable, as the reported incidences were within the range of previous large Phase 3 gastric cancer trials.Additionally, four clinical trials were conducted on ramucirumab for the treatment of metastatic breast cancer, an off-label indication.Three of the four studies had negative results and were associated with 1098 Grade 3-5 adverse events.These findings align with previous research by Carlisle, et al. who observed a similar pattern in the development of two cancer drugs, imatinib and sunitinib, where efforts to apply these drugs to new indications led to repeated negative results and increased burden on trial participants. 11,12The financial and toxicity burden associated with ramucirumab calls for future studies to focus on refining patient selection to maximize the benefit from this treatment.
Our data highlights the ethics of continuing off-label trials in light of negative results.While there are circumstances where off-label use is justifiable, such as in compassionate care cases and exhausted lines of treatment situations, the patient burden, research waste, and overall risk/benefit profile of a drug must be considered.Although clinical trials in oncology inherently involve risk for patients, researchers must continually ensure the potential benefits outweigh the harms.Our results indicate when two non-FDA-approved trials were conducted limited efficacy was observed.This presents an ethical dilemma where patients, in hopes of gaining access to treatments which could improve their health, are more willing to participate in riskier trials. 30These harms directly upset the risk/benefit analysis of a therapeutic, which may be additionally amplified when assessing newer drugs given shorter time frames of most trials may not capture the true toxicity profile. 31Providing patients with therapeutics that may improve their outcome should be the goal for clinical trials.Trialists should strive to uphold these standards and closely examine efficacy outcomes before proceeding to conduct additional studies on the same indication.

| Strengths and limitations
Our study has several strengths.First, investigators received data extraction training and conducted the study   in a masked, duplicate fashion in accordance with the Cochrane Collaboration Guidelines. 32Second, we used Rayyan, a reliable search platform for systematic reviews, to conduct the title and abstract screen. 33Lastly, we uploaded our extraction form, data sheet, and protocol to OSF, an open-source repository that promotes transparency in research. 14Despite being methodologically rigorous our study is not without limitations.First, our initial systematic search may not have yielded all relevant publications.In addition, although we applied measures to minimize errors in data extraction, there is a possibility that mistakes persisted.Lastly, our study cannot be applied to other medications or fields of medicine as it is a cross-sectional design.

| CONCLUSION
We sought to quantify the risk/benefit profile of ramucirumab clinical trials and characterize areas of excess burden.Overall, FDA-approved indications had better efficacy outcomes than non-approved indications.However, a concerning number of adverse events were observed across all trials, even those that led to FDA approval.Participants in ramucirumab randomized controlled trials saw meager gains of only 1.5 months in overall survival when evaluated against a comparison group.Given the toxicity burden of ramucirumab, combined with poor survival gains, clinicians should weigh the potential benefits against the potential risks and adverse events.Future research and evaluations are essential to obtain a more thorough understanding of the optimal utilization and potential risks associated with ramucirumab treatment.

T A B L E 1
Characteristics of included trials.

a
Response rate median calculations for those indications using RECIST or mRECIST only.b Did not measure outcome, failed to report outcomes, or failed to report outcome measure for all enrolled patients.| 7 of 17 KHAN et al.

T A B L E 3 aa
Monotherapy trial characteristics and outcomes by indication.Response rate median calculations for those indications using RECIST or mRECIST only.bDid not measure outcome, failed to report outcomes, or failed to report outcome measure for all enrolled patients.T A B L E 4 Combination therapy trial characteristics and outcomes by indication.Response rate median calculations for those indications using RECIST or mRECIST only.bDid not measure outcome, failed to report outcomes, or failed to report outcome measure for all enrolled patients.T A B L E 5 ∆ PFS and OS for randomized control trials.
C Cu u u ul la at ti iv ve e A Ad dv ve er rs se e E Ev ve en nt ts s Adverse Events Participants C Cu u u ul la at ti iv ve e A Ad dv ve er rs se e E Ev ve en nt ts s a an nd d a ar rt ti i i ip pa an nt ts s o of f R Ra a u u i ir ru u a ab b T Tr ri ia al ls s O Ov ve er r T Ti i e e AE spike due to RAINFALL FDAapproved trials AE spike due to REVEL, RAINBOW, RAISE and REACH FDA-approved trials and ROSE/TRIO-12 (off-label trial)

F I G U R E 3
Cumulative adverse event rate and cumulative ORR.

F I G U R E 4
AERO diagram representing trial progression.F I G U R E 5 Cumulative Grade 3-5 events and cumulative ORR in Phases 1-3 of combination trials.
Cumulative Grade 3-5 events and cumulative ORR in Phases 1-3 of monotherapy trials.